Scalable Processors in the Billion-Transistor Era: IRAM

T he importance of an efficient memory system is increasing as fabrication processes scale down, yielding faster processors and larger memories. This trend widens the processor-memory gap. Not long ago, off-chip main memory was able to supply the CPU with data at an adequate rate. Today, with processor performance increasing at a rate of about 60 percent per year and memory latency improving by just 7 percent per year, 1 it takes dozens of cycles for data to travel between the CPU and main memory. Designers are investing vast amounts of chip resources to bridge this gap. An increasing fraction of the area budget within microprocessor chips is devoted to static RAM (SRAM) caches. For instance, almost half of the die area in the Digital Alpha 21164 is occupied by caches, used solely for hiding memory latency. This cache memory is just a redundant copy of information that would not be necessary if main memory had kept up with processor speed. Still, some applications show poor locality, resulting in low performance even with large caches. 2 Other latency tolerance techniques include combining large caches with some form of out-of-order execution and speculation. Yet this is not an efficient solution for the future, as it requires a disproportionate increase in chip area and complexity. Consider, for example, the MIPS R5000 and R10000 processors. The first is a simple RISC processor, while the second is a complex, out-of-order, speculative one. The R10000 takes 3.43 times more area than the R5000, but its performance, as measured by its SPECint95 peak rating, is only 1.64 times higher. Other architecture alternatives, like wide superscalar and VLIW (very long instruction word), suffer from drawbacks—implementation complexity, low utilization of resources, and immature compiler technol-ogy—or deliver only modest performance improve– ments. Moreover, they usually exacerbate the main memory bandwidth bottleneck. 3 Beyond the uniprocessor, the possibility exists for the integration of multiple processors on a single die, but this integration would place even greater demands on the memory system. For any given die size, putting more processors on a die will result in less on-chip memory for each, thus increasing the number of slow, off-chip memory accesses. In general, increasing the computing resources without a corresponding increase in the on-chip memory will lead to an unbalanced system. Functional units will often be starved for data because of the high latency and limited bandwidth to and from off-chip memory. …